Geo-relevance for images

ABSTRACT

Images may be sorted and categorized by defining a frustum for each image and overlaying the frustums in two, three, or four dimensions to create a density map and identify points of interest. Images that contain a point of interest may be grouped, sorted, and categorized to determine representative images of the point. By including many images from different sources, common points of interest may be defined. Points of interest may be defined in two or three Euclidian dimensions, or may include a dimension of time.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation application of U.S. application Ser.No. 11/867,053, filed on Oct. 4, 2007, the disclosure of which isincorporated herein by reference in its entirety.

BACKGROUND

Organizing and sorting images is a very difficult task. Many websitesenable users to post their own photographs or images and often allowthem to tag the images with a description or other metadata. However,the tags are often very general and difficult to search or furthercategorize. Many users do not take the time to adequately categorizetheir images, and each user may be inconsistent with theircategorization. Because each user may categorize their images in adifferent manner from other users, grouping and organizing largequantities of images can be impossible.

When all the sources of images is combined, the volume of images thatmay contain the same content, such as an iconic landmark like the EiffelTower, can be staggering. When all of the images are availableelectronically, sorting and categorizing the images to identifyimportant landmarks and select a representative image for each landmarkcan be very difficult.

SUMMARY

Images may be sorted and categorized by defining a frustum for eachimage and overlaying the frustums in two, three, or more dimensions tocreate a density map and identify points of interest. Images thatcontain a point of interest may be grouped, sorted, and categorized todetermine representative images of the point of interest. By includingmany images from different sources, common points of interest may bedefined. Points of interest may be defined in two or three Euclidiandimensions, or may include a dimension of time.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings,

FIG. 1 is a diagram illustration of an embodiment showing a system foranalyzing images.

FIG. 2 is a diagram illustration of an embodiment showing a frustum intwo dimensions.

FIG. 3 is a diagram illustration of an embodiment showing a frustum inthree dimensions.

FIG. 4 is a diagram illustration of an embodiment showing a relevancemap created by overlapping several frustums.

FIG. 5 is a flowchart illustration of an embodiment showing a method foranalyzing images.

FIG. 6 is a flowchart illustration of an embodiment showing a method fordetermining frustums.

DETAILED DESCRIPTION

Large numbers of images may be grouped together based on overlappingimage frustums to identify areas of interest. From those areas ofinterest, representative images may be selected. The areas of interestand representative images may be determined merely from the aggregationand analysis of many images in an automated fashion, without a human tointervene and group, classify, or select images.

Each image taken from a camera can be represented by a frustum having apoint of origin, direction, viewing angle, and depth. Components of afrustum definition may be determined precisely using geopositionaldevices such as a GPS receiver, or may be approximately determined by auser selecting an approximate position from where an image was taken andthe approximate direction. Other techniques may also be employed,including stitching images together by mapping one portion of a firstimage with a portion of a second image.

When the frustums of many images are overlaid on a map, areas of highand low frustum density may emerge. In many cases, important buildings,situations, locations, people, or other items may be captured by imagesfrom many different photographers or by many different images. Byexamining a large quantity of images, high density areas may indicate animportant item that has been captured.

Once an area of interest is determined from the density of overlappingfrustums, those images that include the area of interest may beidentified and grouped. In some cases, the group may be analyzed todetermine subgroups.

Various analyses may be performed on the grouped images, such as findingan image that best covers the area of interest or that has the bestsharpness, resolution, color distribution, focal length, or any otherfactor.

The frustums may have various weighting functions applied. In somecases, the weighting function may vary across the viewable area of theimage. In other cases, some images may be weighted differently thanothers.

The analysis may take images from many different sources andautomatically determine areas of interest. The areas of interest may beranked based on the density of coverage and representative imagesselected. The process may be performed without any knowledge of thesubject matter or the likely candidates for important features that maybe within the images. From an otherwise unassociated group ofphotographs or images, the important images may be automaticallydetermined by finding those features or items that are most commonlyphotographed.

In some embodiments, the analysis may be performed with the addeddimension of time. Such analyses may be able to highlight a particularevent or situation.

Some analyses may identify outlying or anomalous images that may havesome importance. As a large number of images are analyzed for ageographic area, a general density pattern may emerge that additionalimages may generally follow. Images that are taken in areas that aremuch less dense may contain items that are ‘off the beaten path’ and maybe also identified as areas of interest.

Throughout this specification and claims, the term ‘image’ is usedinterchangeably with ‘photograph’, ‘picture’, and other similar terms.In many of the methods and operations described in this specification,the ‘image’ may be an electronic version of an image, such as but notlimited to JPEG, TIFF, BMP, PGF, RAW, PNG, GIF, HDP, XPM, or other fileformats. Typically, an image is a two dimensional graphic representationthat is captured using various types of cameras. In some cases, a cameramay capture and create an electronic image directly. In other cases, animage may be captured using photographic film, transferred to paper,then scanned to become an electronic image.

Throughout this specification, like reference numbers signify the sameelements throughout the description of the figures.

When elements are referred to as being “connected” or “coupled,” theelements can be directly connected or coupled together or one or moreintervening elements may also be present. In contrast, when elements arereferred to as being “directly connected” or “directly coupled,” thereare no intervening elements present.

The subject matter may be embodied as devices, systems, methods, and/orcomputer program products. Accordingly, some or all of the subjectmatter may be embodied in hardware and/or in software (includingfirmware, resident software, micro-code, state machines, gate arrays,etc.) Furthermore, the subject matter may take the form of a computerprogram product on a computer-usable or computer-readable storage mediumhaving computer-usable or computer-readable program code embodied in themedium for use by or in connection with an instruction execution system.In the context of this document, a computer-usable or computer-readablemedium may be any medium that can contain, store, communicate,propagate, or transport the program for use by or in connection with theinstruction execution system, apparatus, or device.

The computer-usable or computer-readable medium may be, for example butnot limited to, an electronic, magnetic, optical, electromagnetic,infrared, or semiconductor system, apparatus, device, or propagationmedium. By way of example, and not limitation, computer readable mediamay comprise computer storage media and communication media.

Computer storage media includes volatile and nonvolatile, removable andnon-removable media implemented in any method or technology for storageof information such as computer readable instructions, data structures,program modules or other data. Computer storage media includes, but isnot limited to, RAM, ROM, EEPROM, flash memory or other memorytechnology, CD-ROM, digital versatile disks (DVD) or other opticalstorage, magnetic cassettes, magnetic tape, magnetic disk storage orother magnetic storage devices, or any other medium which can be used tostore the desired information and which can accessed by an instructionexecution system. Note that the computer-usable or computer-readablemedium could be paper or another suitable medium upon which the programis printed, as the program can be electronically captured, via, forinstance, optical scanning of the paper or other medium, then compiled,interpreted, of otherwise processed in a suitable manner, if necessary,and then stored in a computer memory.

Communication media typically embodies computer readable instructions,data structures, program modules or other data in a modulated datasignal such as a carrier wave or other transport mechanism and includesany information delivery media. The term “modulated data signal” means asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal. By way of example,and not limitation, communication media includes wired media such as awired network or direct-wired connection, and wireless media such asacoustic, RF, infrared and other wireless media. Combinations of the anyof the above should also be included within the scope of computerreadable media.

When the subject matter is embodied in the general context ofcomputer-executable instructions, the embodiment may comprise programmodules, executed by one or more systems, computers, or other devices.Generally, program modules include routines, programs, objects,components, data structures, etc. that perform particular tasks orimplement particular abstract data types. Typically, the functionalityof the program modules may be combined or distributed as desired invarious embodiments.

FIG. 1 is a diagram of an embodiment 100 showing a system for analyzingimages. Embodiment 100 is an example of a system that may accept a setof images, determine frustums for each of the images that may be definedby an origin, direction, and field of view as defined in space. Thefrustums may be mapped onto the same two dimensional, three dimensional,or four dimensional space to form a relevance map. The map may showvarious densities of coverage by the images, and the densities may beused to determine areas of interest, which may be used to group theimages together.

The diagram of FIG. 1 illustrates functional components of a system. Insome cases, the component may be a hardware component, a softwarecomponent, or a combination of hardware and software. Some of thecomponents may be application level software, while other components maybe operating system level components. In some cases, the connection ofone component to another may be a close connection where two or morecomponents are operating on a single hardware platform. In other cases,the connections may be made over network connections spanning longdistances. Each embodiment may use different hardware, software, andinterconnection architectures to achieve the functions described.

Embodiment 100 is a mechanism that organizes and sorts images byclassifying image content based on the volume or density of imagecoverage for a certain object, event, or other item. Areas that have themost image coverage may be identified as objects or areas of interest.

For example, when people visit Paris, there are several landmarks thatare often photographed, such as the Eiffel Tower. If several visitors toParis collected their photographs or images and were analyzed byembodiment 100, the Eiffel Tower would likely be one of the mostphotographed objects.

Images 102 are sorted by determining at least an approximate frustumthat defines the image in space. A frustum is a truncated pyramid shapethat may define the viewable area of a photograph or other image. Thefrustum engine 104 may define a frustum 106 through many differentmechanisms.

In some embodiments, a camera may be outfitted with one or more devicesthat may capture metadata about an image. For example, a camera maycapture a focal length, f-stop, view angle, lens size, zoom depth,shutter speed, or any other parameter about the image. Some cameras maybe outfitted with global positioning system (GPS) receivers that mayoutput a globally defined position and, in some cases, a direction forthe image. Some cameras may also track the date and time of the image.

Much of this metadata may be used to reconstruct or partiallyreconstruct a frustum that defines an image. In some embodiments, a usermay locate a position and direction for each image on a map. In otherembodiments, automated algorithms may identify features in an image andattempt to ‘stitch’ or place the image in relationship to another image.From such algorithms, a two or three dimensional model may beconstructed from several two dimensional images.

In some cases, a frustum may be defined only approximately and withlittle precision. For example, a user input may define an image as beingtaken from a particular street corner facing a general direction. As thevolume of images increases, the density maps that may be created fromthe images may be less dependent on the precision of each frustumdefinition.

The various frustums 106 may be fed into a mapping engine 108 that iscapable of generating a density or relevance map 110. The mapping engine108 may effectively overlay the various frustums 106 in relation to eachother.

As in the example above of visitors to Paris, the volume of images thatare taken of the Eiffel Tower may be higher than other places aroundParis. By overlaying a frustum for each image, a high density offrustums may relate to the position of the Eiffel Tower. In someembodiments, the analysis system of embodiment 100 may have no specialknowledge of what portions of an image may be relevant or interesting,but can identify the areas merely based on the volume of images thatpoint to or cover a specific geographic area. Using the system ofembodiment 100 without any outside knowledge, those items that are moreoften photographed would be more densely covered and thus more relevant.

The map analyzer 112 may take the relevance map 110 and identify theareas of interest 114. In many cases, the areas of interest maygenerally be the areas that are most densely covered.

The relevance map 110 may be defined in several different ways. In asimple and easy to understand form, the relevance map 110 may be a twodimensional map on which a triangle representing each image may beoverlaid. Each triangle may represent the physical area captured by animage.

The relevance map 110 may be defined in three dimensions, which mayinclude X, Y, and Z axes. In such a case, each image may be defined by apyramidal frustum. In a simplified embodiment, an image may berepresented as a ray having an origin and direction. For the purposes ofthis application and claims, the term ‘frustum’ shall include a threedimensional pyramidal frustum as well as a triangle, truncated triangle,parallelogram, ray, vector, or other representation of the coverage areaor volume of an image.

In another embodiment, the relevance map 110 may be defined with a timeaxis. By analyzing the relevance map 110 with respect to time, specificevents may be located from the images.

For example, several photographers may take pictures during a weddingreception. During certain events during the wedding reception, such ascutting the cake or a first dance, many of the photographs may be taken.At other times, the number of images may be much fewer. By looking atthe density of images taken over time, an event can be identified. Therelative importance of the event may be determined by the density ofimages captured at that time.

The map analyzer 112 may identify areas of interest 114 using manydifferent mechanisms. In some embodiments, the distribution of valuesacross a relevance map 110 may be analyzed to determine areas or pointsof high values and low values. In some cases, an area of interest may bedefined as a single point. In other cases, an area of interest may bedefined as a defined area or volume.

An area of interest may be identified as an area or point that iscaptured by many images. When many different images are analyzed,especially when the images come from many different sources, there maybe certain items that are photographed more often than others. Suchitems may be the icons, highlights, tourist destinations, or otherimportant features, such as the Eiffel Tower in the example of Parisgiven above.

A grouping engine 116 may create image groups 118 and 120 based on theareas of interest 114. The grouping engine 116 may take an area ofinterest 114 as defined by a point or area, and identify those imageswhose frustums overlap at least a portion of the area of interest. Insome cases, an image frustum may cover only a small portion of the areaof interest.

After the images are grouped, a representative image 122 and 124 may beselected from the groups 118 and 120, respectively. A representativeimage may be automatically determined using several criteria. Onecriterion may be to select an image that has a frustum that capturesmost of the area of interest and where the area of interest fills thefrustum. Using the Eiffel Tower example, such a criterion may preferimages in which the Eiffel Tower is centered in the image and fullyfills the image, and may exclude images where the Eiffel Tower is off toone side or is a much smaller portion of the image.

Other criteria may be additionally considered to select a representativeimage, including the sharpness of the image, resolution, contrast, orother criteria. Some embodiments may perform additional analysis toensure that the object of interest is included in the representativeimage 118 or 120. In some cases, the additional analysis may involvehuman operators to select a single representative image from a group ofprospective images, or automated tools may be used to analyze the imagesto determine if the object of interest is actually portrayed in therepresentative image.

The grouping engine 116 may form groups of images that contain specificareas of interest, and may leave many images unclassified. In somecases, the grouping engine 116 may create a hierarchical tree ofclassification. Such a tree may be formed by creating a first group ofimages, then analyzing the group of images to identify second-levelareas of interest within the group, and creating sub-groups based on thesecond-level areas of interest. In many embodiments, a vast number ofimages may be processed in a recursive manner to generate a deep,multilevel hierarchical tree of areas of interest.

The embodiment 100 may be a computer application that operates on aprocessor 126. In some cases, the various functions of the frustumengine 104, mapping engine 108, map analyzer 112, and grouping engine116 may be software, hardware, or combination of software and hardwarecomponents that are capable of performing the functions described forthe respective items. In some embodiments, one or more of the frustumengine 104, mapping engine 108, map analyzer 112, and grouping engine116 may be performed by separate devices and may be performed atdifferent times.

For example, many functions of the frustum engine 104 may beincorporated into a camera, where a GPS receiver may determine a frustumorigin and direction and a field of view and focal length may bedetermined from lens settings for the camera.

In other cases, the frustum engine 104 may be performed by two or moresystems, such as when a user uploads an image to a website and uses amap on the website to input the origin and direction of the image. Asecond system may process the information to calculate an approximatefrustum for the image.

Some embodiments may use a high powered processor, server, cluster,super computer, or other device to perform the functions of a mappingengine 108 and map analyzer 112. The functions of the mapping engine 108and map analyzer 112 may involve processing vast amounts of data andperforming many computations. In some cases, thousands or even millionsof images may be processed to determine areas of interest, especiallywhen a detailed and deep multilevel hierarchical analysis is performed.

In some cases, a new image may be analyzed using embodiment 100 after alarge number of images have been processed and a relevance map 110already exists. In such an analysis, a new image may be compared to anexisting relevance map 110 and areas of interest 114 by the groupingengine 116 so that the new image may be classified.

As the number of images that are processed increases, the areas ofinterest 114 may become stable enough that additional images may notchange the areas of interest 114 significantly. In some cases, groups oftens, hundreds, or thousands of images may be analyzed to determineareas of interest 114, from which many other images may be classifiedand grouped. In other cases, hundreds of thousands or even millions ofimages may be processed to produce a deep hierarchical tree of areas ofinterest. Once such a tree is defined, new images may be quicklyclassified into various groups by the grouping engine 116.

FIG. 2 is a diagram representation of an embodiment 200 showing a twodimensional representation of a frustum. In two dimensions, a frustumbecomes a parallelogram.

The frustum 200 has an origin 202 and direction 204. The field of view206 may be centered about the focal distance 208 and may be defined by adepth of field 210. In some cases, the frustum 200 may be determinedusing a view angle 212 or other lens parameters.

The frustum 200 may be an approximation of the physical area that may becaptured by an image. The physical area may be the approximate area thatwould be in focus based on the various parameters of the specific camerathat captured the image or of a general camera. In some cases, a cameramay be able to capture some image-specific metadata such as focallength, f-number or focal ratio, aperture, view angle, or otherparameters. In other cases, an image may be approximated by assuming astandard view angle, focal length, or other parameters when someparameters are not specifically available.

The origin 202 may be the approximate position of a camera when an imageis captured, and the direction 204 may be centerline of the lens of thecamera. The view angle 212 may be defined by the focal length of thelens. The depth of field 210 may be defined by the lens aperture.

In many cases, a focal distance may be infinity, which may be common foroutdoor pictures. If this were literally interpreted, the frustum may beinfinitely deep. In practice, each image has an object that defines thefurthest point that the frustum actually captures, and the frustum edgefurthest from the origin may follow a contour of the objects in theimage.

In some embodiments, the far edge of the frustum may be mapped to a twodimensional or three dimensional representation of the physical locationnear the place where a picture was taken. For example, if a photographerwere standing on a street corner in Paris taking a picture across aneighborhood but pointed in the direction of the Eiffel Tower, the imagemay have tall buildings as a backdrop and may not include the EiffelTower at all. If a frustum were created to represent the image, the faredge of the frustum may be mapped to a representation of the buildingsin the neighborhood and thus the frustum may be truncated so that itdoes not include the area of the Eiffel Tower.

The frustum 200 is a two dimensional representation of an image. In manyembodiments, a two dimensional approximation of each image may besufficient to determine areas of interest and classify images.

FIG. 3 is a diagram representation of an embodiment 300 showing a threedimensional representation of a frustum. In three dimensions, thefrustum 300 may be a pyramidal frustum.

The frustum 300 may have an origin point 302 and direction vector 304,and the depth of field 308 may define the field of view frustum 306.

The frustum 300 may be defined using the same parameters as discussedfor the two-dimensional frustum illustrated in FIG. 2. The frustum 300illustrates a three dimensional volume that may capture the items infocus for a particular image.

In some embodiments, a three dimensional representation of each imagemay be used to determine points, areas, or volumes of interest. Theadditional complexity of determining a vertical component for thedirection vector 304 may be difficult to assess in some embodiments, butmay add a more accurate identification of areas of interest.

FIG. 4 is a diagrammatic illustration of an embodiment 400 showing a setof overlapping frustums used for mapping. Three frustums are overlappedto illustrate how an area of interest 420 may be determined. In apractical embodiment, frustums from many more images may be used, butthe embodiment 400 is chosen as simplified illustration.

A first frustum 402 is defined by an origin 404 and a direction 406. Asecond frustum 408 is likewise defined by an origin 410 and direction412. Similarly, a third frustum 414 is defined by an origin 416 anddirection 418.

Each of the frustums 402, 408, and 414 may be defined by applyingstandard parameters for an image, by determining parameters from acamera when an image is taken, by user input, or by other mechanisms.

The frustums 402, 408, and 414 are placed relative to each other on acoordinate system to create the diagram 400. The area of interest 420 isthe area that is overlapped by all three frustums.

An object 422 may be located within the area of interest 420. Inembodiment 420, the object 422 is photographed from three differentangles and may present three very different visual images at each angle.By overlapping the image frustums in space, the images represented byfrustums 402, 408, and 414 may be categorized and grouped togetherproperly even though the object 422 may look very different in eachview.

In some cases, a density map may be used to locate areas of interest,and may be termed a relevance map in some instances. The density may bedetermined by the number of overlapping frustums over a particular areaor point. For example, the area of interest 420 has a density of threein embodiment 400. The area of interest 420 is the highest density inembodiment 400 and thus may be selected as the area of interest.

In determining the relevance map of embodiment 400, each frustum may beassigned a value of one. In some embodiments, a frustum may have adensity or relevance function that may be applied across the area of thefrustum. One example of a relevance function may be to apply a standarddistribution curve across the frustum. Another example of a relevancefunction may be to determine a contrast ratio or other type ofevaluation from the image contents. Such a relevance function may becalculated individually for each image.

The relevance function may be used to vary the importance of areas of afrustum when determining a density map or relevance map. In many cases,an image of an object, such as the Eiffel Tower, will have the objectcentered in the photograph or at least in the center portion of theimage. In many cases, an image frustum may have greater importance orsignificance in the center portion of an image and lesser importance atthe edges. Thus, a relevance function may be applied to the frustum insome embodiments to more accurately use data derived from the image,such as the contrast ratio or other computation performed on the imageitself, or to use an approximation of a standard distribution.

In some cases, a frustum may have a density value that is greater orlesser than another frustum. For example, images taken with snapshottype camera devices may be given less importance than images taken withhigh resolution, professional cameras. Images with high resolution orhigher contrast may be given more weight than lower resolution or lowercontrast, or images taken during daytime may be given more weight thannighttime images. Newer images may be given more weight than older ones,or vice versa.

FIG. 5 is a flowchart illustration of an embodiment 500 showing a methodfor analyzing images. Embodiment 500 is one method by which a group ofimages may be analyzed by creating a map from image frustums,determining areas of interest based on the map, and grouping imagesaccording to the areas of interest. The images may be ranked within thegroups as well.

Images are received in block 502. The images may be any type of images,photographs, video frames, or other captured images. In many cases, theimages may be conventional visual images, but images from ultraviolet,infrared, or other image capture devices may also be used.

For each image in block 504, a frustum may be determined in block 506.FIG. 6 included hereinafter will discuss various mechanisms andtechniques for determining a frustum for an image.

The frustums are mapped in block 508. In many embodiments, a map may becreated by orienting each frustum relative to each other in a geographicrepresentation. In some cases, a two dimensional representation may beused, while in other cases, a three dimensional representation may beused.

Some maps may include a timeline component. Such maps may map thefrustums in a two dimensional plane or in three dimensional space, aswell as mapping the images in a time dimension.

In many embodiments, frustums may be mapped with a consistent value foreach frustum. In some cases, frustums may be weighted due to contentthat may be derived from the image itself, such as image resolution,sharpness, contrast, or other factors, or frustums may be weighted dueto other factors such as image sources, time of day, or other metadata.

Some embodiments may apply a constant value for each frustum, such asapplying a designated value for the entire volume or area represented bythe frustum. Some embodiments may apply a function across the volume orarea represented by the frustum so that some portion of the image may beweighted higher than another.

In such embodiments, a weighting function may be defined for a frustumbased on information derived from the image itself, such as determiningareas of the image that have more detail than others, or a function maybe applied that weights one area of a frustum higher than another, suchas a curved distribution across the frustum.

Areas of interest may be determined in block 510 by identifying volumes,areas, or points within the map that have a high or low concentration offrustum coverage. The area of interest may be defined in two dimensionalor three dimensional space. In cases where a time element is considered,an area of interest may also include a factor on a time scale.

For each area of interest in block 512, and for each image in block 514,the coverage of the area of interest by the image is determined in block516. If the coverage is zero in block 518, the image is skipped in block520. If the image frustum covers at least part of the area of interestin block 518, the image is added to the group of images associated withthe area of interest in block 522.

After grouping the images in blocks 514-522, the images may be rankedwithin the group in block 524. The ranking criteria may be severalfactors, including degree of image coverage, how well the area ofinterest is centered within the image, the amount of overlap of theimage frustum to the area of interest, image resolution, imagesharpness, and other factors.

A representative image may be selected in block 526. In some cases, thehighest ranked image may be selected. In other embodiments, a humanoperator may select a representative image from several of the topranked images.

FIG. 6 is a flowchart diagram of an embodiment 600 showing a method fordetermining a frustum. The embodiment 600 illustrates several differenttechniques that may be used to determine a frustum for an image. Thefrustum combines geopositional information from one or many sources todetermine a size, position, and orientation of a frustum in space.

Frustum determination begins in block 602.

If the camera used to capture an image has a GPS receiver and is GPSenabled in block 604, the GPS location of the camera at the time animage is captured is used as the frustum origin in block 606. If the GPSfeature is also capable of giving a direction for the image in block608, the GPS direction is used for the frustum in block 610.

If the image is captured without GPS information in block 604, thegeneral location of an image may be retrieved from user suppliedmetadata in block 612. The user supplied metadata may be tags,descriptions, or any other general location information. In block 612,the general location may be non-specific, such as ‘Paris, France’, or‘around the Eiffel Tower’.

If an automated analysis tool is available in block 614, locationalanalysis may be run on an image database in block 618. Locationalanalysis may attempt to map an image to other images to build a two orthree dimensional representation of the objects in the image.

In many such techniques, the automated analysis tool may ‘stitch’ imagestogether by mapping a first portion of a first image with anotherportion of a second image. When the two images are stitched together,the frustums defining each image may be defined with respect to eachother. When enough images are stitched together, a very complete modelof the objects in the images can be determined. In many cases, aby-product of such analyses is a very accurate frustum definition foreach image.

If no automated tool is available in block 614, a user may input anapproximate location on a map for the image in block 616. For example, auser may be shown an interactive map on which the user may select apoint and direction where the user was standing with a camera when animage was taken. Other types of user interfaces may be used to captureapproximate origin and direction information that may be used todetermine an approximate frustum for an image.

Additional metadata concerning an image may be gathered in block 620.Such metadata may include various parameters about the camera when animage was taken, such as f-stop, aperture, zoom, shutter speed, focallength, ISO speed, light meter reading, white balance, auto focus point,or other parameters. Some of the parameters may be used directly incalculating or determining a frustum. Other parameters may be used inlater analysis or classification of images. In some cases, a date andtime stamp for each image may be also gathered.

Some metadata may be derived from the image itself. For example,resolution, color distribution, color density, or other metadata may bederived and stored.

In the preceding steps, at least an approximate origin and direction maybe determined for a frustum. In block 622, a focal point may bedetermined. In some cases, the focal point may be derived from thevarious camera settings or from estimating the distance from an imageorigin to a known landmark. In many cases, a camera's focal point may beinfinity or close to infinity for distances greater than 50 or 100 feet.

The depth of field may be determined in block 624. In many cases, thedepth of field may be a function of the amount of light, aperture,focus, and other photographic parameters. In cases where the parametersare not known, a standard parameter may be assumed.

The frustum definition may be determined in block 626. In some cases, avery precise frustum may be calculated using GPS inputs and detailedparameters taken from a camera when an image is created. In other cases,a frustum may be a gross approximation based on coarse location anddirection information input by a user along with a standardized orgeneralized frustum size that may be assumed by default.

In many embodiments, the precision of a frustum definition may notadversely affect the resultant relevance map or alter the areas ofinterest that may be chosen from the relevance map, especially when avery large number of images are processed.

The foregoing description of the subject matter has been presented forpurposes of illustration and description. It is not intended to beexhaustive or to limit the subject matter to the precise form disclosed,and other modifications and variations may be possible in light of theabove teachings. The embodiment was chosen and described in order tobest explain the principles of the invention and its practicalapplication to thereby enable others skilled in the art to best utilizethe invention in various embodiments and various modifications as aresuited to the particular use contemplated. It is intended that theappended claims be construed to include other alternative embodimentsexcept insofar as limited by the prior art.

What is claimed is:
 1. A computer system having a processor and a memorycoupled to the processor, the memory containing instructions, whenexecuted by the processor, causing the processor to perform a methodcomprising: determining a plurality of physical areas in space coveredby one or more cameras carried by one or more users, each of thephysical areas defined by an approximate point of origin and anapproximate direction; overlapping the determined plurality of physicalareas relative to one another in a geographic representation, theoverlapped physical areas having a density of coverage; identifying anarea of interest based on the density of coverage of the overlappedphysical areas; and associating one of the users with the area ofinterest if the camera carried by the user has a physical area that atleast partially overlaps the area of interest.
 2. The computer system ofclaim 1 wherein: the area of interest is associated with at least one ofa structure, an object, or a tourist destination; and the method furtherincludes associating one of the users with the at least one ofstructure, object, or tourist destination if the camera carried by theuser has a physical area that at least partially overlaps the area ofinterest.
 3. The computer system of claim 1 wherein: the area ofinterest is associated with at least one of a structure, an object, or atourist destination; the method further includes: associating a firstone of the users with the at least one of structure, object, or touristdestination if a first camera carried by the first user has a firstphysical area that at least partially overlaps the area of interest; andassociating a second one of the users with the at least one ofstructure, object, or tourist destination if a second camera carried bythe second user has a second physical area that at least partiallyoverlaps the area of interest; and wherein the first physical area ofthe first camera at least partially overlaps the second physical area ofthe second camera.
 4. The computer system of claim 1 wherein: the areaof interest is associated with at least one of a structure, an object,or a tourist destination; the method further includes: associating afirst one of the users with the at least one of structure, object, ortourist destination if a first camera carried by the first user has afirst physical area that at least partially overlaps the area ofinterest, the first camera having a first approximate point of originand a first approximate direction; and associating a second one of theusers with the at least one of structure, object, or tourist destinationif a second camera carried by the second user has a second physical areathat at least partially overlaps the area of interest, the second camerahaving a second approximate point of origin and a second approximatedirection; and wherein the first approximate point of origin isdifferent than the second approximate point of origin, and/or the firstapproximate direction is different than the second approximatedirection.
 5. The computer system of claim 1 wherein: the area ofinterest is associated with at least one of a structure, an object, or atourist destination; the method further includes: associating a firstone of the users with the at least one of structure, object, or touristdestination if a first camera carried by the first user has a firstphysical area that at least partially overlaps the area of interest, thefirst camera having a first approximate point of origin and a firstapproximate direction; and associating a second one of the users withthe at least one of structure, object, or tourist destination if asecond camera carried by the second user has a second physical area thatat least partially overlaps the area of interest, the second camerahaving a second approximate point of origin and a second approximatedirection; and wherein the first physical area of the first camera atleast partially overlaps the second physical area of the second camera,and wherein the first approximate point of origin is different than thesecond approximate point of origin, and/or the first approximatedirection is different than the second approximate direction.
 6. Thecomputer system of claim 1 wherein identifying the area of interestincludes identifying an area on the geographic representation that ismost densely covered as the area of interest.
 7. The computer system ofclaim 1 wherein identifying the area of interest includes: calculating adistribution of density of coverage on the geographic representation;and identifying an area on the geographic representation as the area ofinterest based on the calculated distribution.
 8. The computer system ofclaim 1 wherein the area of interest is a first area of interest, andwherein identifying the area of interest includes: calculating adistribution of density of coverage on the geographic representation;and identifying a second area of interest based on the calculateddistribution, the second area of interest being different than the firstarea of interest.
 9. A computer-implemented method for identifying anarea of interest, the method comprising: determining a plurality offrustums individually associated with a field of view of individualcameras, each of the frustums having an approximate point of origin andan approximate direction, wherein the individual frustums represent ageographic coverage area or volume of the corresponding cameras inspace; arranging the plurality of frustums relative to one another basedon the approximate point of origin and the approximate direction in ageographic representation; identifying a density of coverage of theplurality of frustums on the geographic representation; and determiningthe area of interest based on the identified density of coverage. 10.The method of claim 9 wherein the cameras are associated with one ormore users, and wherein the method further includes: determining if oneof the cameras has a frustum that at least partially overlaps thedetermined area of interest on the geographic representation, the camerabeing associated with one of the users; and if the frustum overlaps thedetermined area of interest, associating the user with the determinedarea of interest.
 11. The method of claim 9 wherein arranging theplurality of frustums includes: representing the individual frustums asat least one of a triangle, a truncated triangle, a parallelogram, aray, or a vector; and overlaying the plurality of the at least one oftriangle, truncated triangle, parallelogram, ray, or vector on a map.12. The method of claim 9 wherein arranging the plurality of frustumsincludes: representing the individual frustums as a triangle or atruncated triangle; and overlaying the plurality of triangles ortruncated triangles on a two-dimensional map.
 13. The method of claim 9wherein arranging the plurality of frustums includes: representing theindividual frustums as a pyramid; and overlaying the plurality ofpyramids on a three-dimensional map.
 14. The method of claim 9 whereinarranging the plurality of frustums includes: arranging the plurality offrustums relative to one another based on the approximate point oforigin, the approximate direction, and time in the geographicrepresentation; wherein identifying the density of coverage includesidentifying a density of coverage with respect to time; and wherein themethod further includes determining an event based on the identifieddensity of coverage with respect to time.
 15. The method of claim 9wherein determining the area of interest includes determining the areaof interest as at least one of a point, an area, or a volume based onthe identified density of coverage.
 16. A computer system foridentifying an area of interest, the computer system comprising: meansfor determining a plurality of physical areas in space covered by one ormore cameras, each of the physical areas having an approximate point oforigin and an approximate direction, and wherein each of the camerasbeing associated with a user; means for arranging the determinedplurality of physical areas relative to one another in a geographicrepresentation, at least some of the physical areas overlap one another;means for selecting the area of interest based on the overlapping of atleast some of the physical areas; means for determining if one of thecameras has a physical area that at least partially overlaps theselected area of interest; and means for associating one of the userswith the area of interest if the physical area of the camera associatedwith the user at least partially overlaps the area of interest.
 17. Thecomputer system of claim 16 wherein the means for arranging thedetermined plurality of physical areas includes means for arranging thedetermined plurality of physical areas as triangles or truncatedtriangles in a two-dimensional geographic representation.
 18. Thecomputer system of claim 16 wherein the means for arranging thedetermined plurality of physical areas includes means for arranging thedetermined plurality of physical areas as pyramids in athree-dimensional geographic representation.
 19. The computer system ofclaim 16 wherein the means for selecting the area of interest includes:means for calculating a distribution of density of coverage of thephysical areas on the geographic representation; and means foridentifying an area on the geographic representation as the area ofinterest based on the calculated distribution.
 20. The computer systemof claim 16 wherein the area of interest is a first area of interest,and wherein the means for selecting the area of interest includes: meansfor calculating a distribution of density of coverage of the physicalareas on the geographic representation; and means for identifying asecond area of interest based on the calculated distribution, the secondarea of interest being different than the first area of interest.